Communication Flows: Analysis of an Email Network

Communications networks as diverse as wired telephone systems and online chat forums record information about every message that they carry, creating enormous, relentlessly growing streams of data. Hidden (often deeply hidden) in these streams is a rich, dynamic view of the people who use the network and their patterns of interaction, but tools for extracting, analyzing and visualizing that information are lacking. Email communications present particularly challenging problems, because even basic information, such as the identities of senders and recipients of a message, must be pieced together by parsing log files on multiple servers across time. People may have multiple aliases that have to be translated to unique identifiers, and lists of group recipients may have to be exploded to individual people. Moreover, the ratio of spam, which is of little use for understanding patterns of communication, to useful messages is high even with aggressive spam filters. This report describes a statistical approach to meeting these challenges for an email history derived from 75 days of log files for several mail servers that handle email for hundreds of people in the research organization at Bell Labs. Some of the results about the structure of the email network are surprising and contradict common assumptions about social network behavior. The email exchanges for a subset of addresses that exchange email with each other and with addresses outside the corporation are analyzed in detail, providing a firm basis for building statistical descriptions of normal email activity that can be used to identify anomalous behavior.